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The Ascent Force and Moment Aerodynamic (AFMA) Databases (DBs) for the Ares I 
Crew Launch Vehicle (CLV) were typically based on wind tunnel (WT) data, with 
increments provided by computational fluid dynamics (CFD) simulations for aspects of the 
vehicle that could not be tested in the WT tests. During the Design Analysis Cycle 3 analysis 
for the outer mold line (OML) geometry designated A106, a major tunnel mishap delayed 
the WT test for supersonic Mach numbers ( M) greater than 1.6 in the Unitary Plan Wind 
Tunnel at NASA Langley Research Center, and the test delay pushed the final delivery of 
the A106 AFMA DB back by several months. The aero team developed an interim database 
based entirely on the already completed CFD simulations to mitigate the impact of the delay. 
This CFD-based database used a response surface methodology based on radial basis 
functions to predict the aerodynamic coefficients for M > 1.6 based on only the CFD data 
from both WT and flight Reynolds number conditions. The aero team used extensive 
knowledge of the previous AFMA DB for the A103 OML to guide the development of the 
CFD-based A106 AFMA DB. This report details the development of the CFD-based A106 
Supersonic AFMA DB, constructs a prediction of the database uncertainty using data 
available at the time of development, and assesses the overall quality of the CFD-based DB 
both qualitatively and quantitatively. This assessment confirms that a reasonable 
aerodynamic database can be constructed for launch vehicles at supersonic conditions using 
only CFD data if sufficient knowledge of the physics and expected behavior is available. 
This report also demonstrates the applicability of non-parametric response surface modeling 
using radial basis functions for development of aerodynamic databases that exhibit both 
linear and non-linear behavior throughout a large data space. 


Nomenclature 


Acronyms and Abbreviations 

AFMA = ascent force and moment aerodynamics 
CFD = computational fluid dynamics 

CLV = crew launch vehicle 

CP = center of pressure 

CxP = Constellation Program 


C2C 

DAC 

DB 

DBI 

DBM 

DOF 

EXP 

FLT 

F&M 

GN&C 

GP 

ITAR 

LaRC 


code-to-code 
design analysis cycle 
database 

database interpolation 
database modeling 
degree(s) of freedom 
experimental 
flight 

force(s) and moment(s) 

Guidance, Navigation and Control 
gimbal point of the first stage 
International Traffic in Arms Regulations 
Langley Research Center 
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MI = margin index 

MRC = moment reference center 

OML = outer mold line 

RBF = radial basis function 

RS = response surface 

SBU = sensitive but unclassified 

UQ = uncertainty quantification 

UPWT = Unitary Plan Wind Tunnel (NASA LaRC facility) 

V&V = verification and validation 

WT = wind tunnel 

Roman Symbols 

C A , CAPF = axial force coefficient in the missile axis system (forebody) 

Cj = generic force or moment coefficient 

C i re f = reference value for any force or moment coefficient 

C/, CLLPF = rolling moment coefficient in the missile axis system (forebody) 

C m , CLMPF = pitching moment coefficient in the missile axis system (forebody) 

C„, CLNPF = yawing moment coefficient in the missile axis system (forebody) 

Cv, CNPF = normal force coefficient in the missile axis system (forebody) 

Cy, CYPF = side force coefficient in the missile axis system (forebody) 

Cl = baseline A106 configuration with all protuberances 

C4 = axisymmetric A106 configuration with all protuberances removed 

d 2 = bias correction factor 

E = denotes an estimated global error between two sources of data 

E„ = denotes an estimated error between two sources of data at a given condition 

Ei.cFDMas = bias term of estimated CFD validation uncertainty 

m = slope term in Reynolds number increment model 

M = Mach number 

N = number of available conditions 

R = range of data (maximum value minus minimum value) 

R 2 = coefficient of determination (goodness-of-fit) 

Re = Reynolds number (per foot), ft 1 

Ui = uncertainty in a coefficient 

Ui, cfd, random = random component of estimate CFD uncertainty 

Ui vai. random = random term of estimated CFD validation uncertainty 

Greek Symbols 

ar, ALPF1AT = total angle of attack (denoted ALPHAT in figures for better readability) 

8 = offset term in Reynolds number increment model 

A = denotes an incremental coefficient or value 

<j> a , PHIA = aerodynamic roll angle (denoted PHIA in figures for better readability) 

I. Introduction 

T HE development and design process of the NASA crew launch vehicle (CLV), Ares I, required that the 
aerodynamics of the vehicles were adequately characterized for all phases of the flight envelope and reasonable 
estimates of the aerodynamic uncertainties were provided. One of the primary deliverables provided by the Ares I 
Aerodynamics Panel throughout the entire design process was the Ascent Force and Moment Aerodynamics 
(AFMA) Database (DB). This DB provided the six degree-of-freedom (DOF) aerodynamic force and moment 
(F&M) coefficients for the CLV during nominal ascent flight from subsonic Mach numbers (M) through transonic 
and supersonic flight before stage separation (Mach numbers from 0.5 to around 5.5) for all total angles of attack 
below 10 degrees. The DB team constructed an AFMA DB from experimental data obtained in a suite of wind 
tunnels (WT) that spanned the entire Mach range for each major design analysis cycle (DAC) of the Ares I CLV. 
The team typically used NASA Langley Research Center’s (LaRC) Unitary Plan Wind Tunnel (UPWT) as the 
primary wind tunnel for the supersonic (M > 1.6) data for the AFMA DBs. The aero team also conducted 
computational fluid dynamics (CFD) simulations and used the CFD data to provide increments for elements that 


2 

American Institute of Aeronautics and Astronautics 



could not be reasonably tested in the wind tunnel, such as simulating the flight (FLT) Reynolds number (Re) 
environment. The CFD data were also used to provide data not practical via experimental means, such as surface 
pressure data on the surface of the CLV and lineloads, and to provide quick turnaround databases and increments for 
small geometry changes. The CFD best practices developed by the Aero Panel for Ares simulations dictated that all 
CFD solutions be obtained before the wind tunnel data. 1 

The CLV outer mold line (OML) geometry for DAC-3 was designated A 106, and the aero team again planned 
and conducted a wind tunnel experiment in the UPWT (titled UPWT Tests 1972 and 1843) to acquire the supersonic 
data for the AFMA DB. 2 Early on during the test, a major mechanical failure in the tunnel created a delay of over 
one month. This delay significantly impacted the project timeline because numerous analyses depend on receiving 
the AFMA DB. The aero team constructed an interim AFMA DB for M > 1.6 using only the available CFD 
simulations to mitigate the effects of this delay. The aero team used a non-parametric response surface (RS) 
generation algorithm based on radial basis functions (RBFs) to construct the interim DB based on the available CFD 
solutions. RBF methodology allowed the team to tailor the final database to model the behavior expected based on 
previous AFMA DB data for the A103 OML. Several months after the CFD-based AFMA DB was released, the 
aero team developed and released the final A106 AFMA DB constructed from the UPWT Test 1972/1843 data. For 
convenience, this paper will refer to the initial DB as the CFD-based Supersonic A106 AFMA DB and the final DB 
as the WT-based Supersonic A106 AFMA DB, or just the CFD-based and WT-based DBs for short. 

The purpose of this report is to present the methodology used to develop the CFD-based DB, develop a model 
for the uncertainty in the CFD-based DB based on data that were available at the time the CFD-based DB was 
developed, and assess the adequacy and accuracy of the CFD-based DB relative to the WT-based DB by comparing 
the results from each DB. Section I gives a brief introduction to the Ares I CLV, the available CFD solutions for the 
A106 OML, and response surface generation using radial basis functions. Section II details the methodology used to 
construct the CFD-based DB, and Section III discusses uncertainty modeling for the CFD-based DB. Section IV 
provides a brief overview of the final WT-based DB development and uncertainty. Section V compares the two 
databases, both qualitatively and quantitatively, to assess the quality and accuracy of the CFD-based DB relative to 
the final WT-based DB. Finally, some conclusions are presented in Section VI. 

A. The Ares I CLV 

Ares I was an in-line, two-stage, crew launch vehicle under development by NASA under the Constellation 
Project (CxP). The Ares I aerodynamics team, spanning several NASA centers, conducted extensive aerodynamic 
testing and computational simulations in support of the overall vehicle integration effort. 3 * ~ 9 Figure 1 shows the 
A106 OML for the vehicle along with definitions for the coordinate axis systems, aerodynamic force and moment 
coefficients, and flow angles. All force and moment coefficients in this report are forebody coefficients, meaning the 
forces on the base of the vehicle have been removed, and are reported with the moment reference center (MRC) at 
the first stage gimbal point (GP), which is approximately the origin of the axes systems in Figure 1. Reference 10, 
which summarizes the uncertainty modeling for the WT-based DB, details the effect of the MRC on uncertainty. 

The Ares I CLV, including its predicted performance and certain other features and characteristics, has been 
defined by the U.S. Government to be sensitive but unclassified (SBU). Information deemed to be SBU requires 
special protection and may not be disclosed to an international audience, such as the audience present at the 2011 
AIAA Applied Aerodynamics Conference. To comply with SBU restrictions, details such as axis scales have been 
removed from some figures in this paper and some results are presented in percent of reference values. It is the 
opinion of the author that these alterations do not invalidate the technical content and impact of this analysis. 
Whenever practical, the same grid scales were used for normal and side force coefficient and pitching and yawing 
moment so that the reader can infer the relative magnitudes of those coefficients. 
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- Freestream flow velocity vector 
X B , Y b , Z b - Body axis vectors 
X p , Y , Z p - Missile axis vectors 

+CNPF +CNF 

A 

,+<P : 


a - Angle of attack 
p - Sideslip angle 
u T - Total angle of attack 
tp - Aerodynamic roll angle 


+CLLF, + y + y 
+CLLPF — l! ’ p 



+CAF, 

+CAPF 



CAF - Axial force coefficient (body axis) 
' CYF - Side force coefficient (body axis) 

CNF - Normal force coefficient (body axis) 
CLLF - Rolling moment coefficient (body axis) 
CLMF - Pitching moment coefficient (body axis) 

CLNF - Yawing moment coefficient (body axis) 

CAPF - Axial force coefficient (missile axis) 

CYPF - Side force coefficient (missile axis) 

CNPF - Normal force coefficient (missile axis) 

CLLPF - Rolling moment coefficient (missile axis) 

CLMPF - Pitching moment coefficient (missile axis) 

CLNPF - Yawing moment coefficient (missile axis) 

Figure 1. Axis system, force and moment coefficient, and flow angle definitions for the Ares I CLV. 


B. Summary of A106 CFD Data 

CFD simulations were a crucial aspect of the aerodynamic analyses performed by the Ares I aero team 
throughout the design process. CFD was used in numerous roles, such as supporting experimental data from wind 
tunnel tests and providing detailed information or simulating conditions not feasible experimentally. This paper does 
not describe the CFD analyses in depth; details on the CFD efforts for Ares I are available elsewhere. 1 6 9 The aero 
team used two CFD codes to support the A106 AFMA DB: USM3D, which is an unstructured, tetrahedral, cell- 
centered, finite volume solver, 11 and OVERFLOW, which is a structured, overset multiblock grid solver. 1 " 
OVERFLOW was considered the primary code for subsonic conditions, and USM3D was the primary code for 
transonic and supersonic computations. Each code repeated selected conditions from the other code’s primary 
regime for code-to-code comparisons. This paper does not present any OVERFLOW solutions because USM3D was 
the primary code for M > 1.6, and the DB team used only USM3D solutions to construct the CFD-based DB. 
Section 111 contains some discussion of code-to-code differences, and more details about the code-to-code 
comparisons from earlier Ares I DACs are reported in reference 13. 

There were a total of 221 CFD solutions from USM3D for the A106 OML at the WT Re available when the 
CFD-based DB was developed, and 245 such solutions at FLT Re. Of these solutions, 104 WT Re and 77 FLT Re 
solutions corresponded to Mach numbers greater than or equal to 1.6. The UQ team requested an additional eight 
solutions (four for M> 1.6) at randomly chosen conditions throughout the domain for assessment of the CFD-based 
AFMA DB. Table 1 below lists the WT Re CFD simulations available for M > 1.6, and Table 2 lists the FLT Re 
solutions. The aero team partially designed the WT Re CFD matrix with response surface generation in mind, 
although the CFD team added numerous conditions to address other needs. The FLT Re matrix was primarily 
designed to address needs other than RS generation. The DB team designed the WT Re CFD matrix using general 
conceptual ideas from design of experiments (DOE), though the final matrix is not any particular DOE design. At 0 
degrees total angle of attack, the aerodynamic roll angle is undefined by definition. For convenience, the team 
assumed an aerodynamic roll angle of 0 degrees for all CFD solutions acquired at total angle of attack of 0 degrees, 
but any other aerodynamic roll angle could have been chosen. 
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Table 1. WT Reynolds number CFD solutions available for M> 1.6 

Mach Number Total Angle of Attack Aerodynamic Roll Angle 

0° (<j) a undefined for a T = 0°) 

0°, 90°, 180°, 270° 

0°, 30°, 150°, 180°, 210°, 345° 

90°, 270° 

0° to 330° in 30° increments 
0° to 330° in 30° increments, 340°, 345°, 350°, 355° 


2.0 

0° 

0° (q undefined for a T = 0°) 

2° 

0°, 90°, 180°, 270° 

4° 

0°, 60°, 150°, 180°, 240°, 345° 


6° 

90°, 270° 

8° 

0°, 90°, 180°, 270° 


0° 

0° (c/) a undefined for a T = 0°) 


2° 

0°, 90°, 180°, 270° 

3.0 

4° 

0°, 60°, 150°, 180°, 240°, 345° 


6° 

90°, 270° 


8° 

0° to 330° in 30° increments 


0° 

0° (<f) a undefined for a T = 0°) 


2° 

0°, 90°, 180°, 270° 

4.5 

4° 

0°, 30°, 60°, 120°, 150°, 180°, 210°, 240°, 300°, 345° 


6° 

90°, 270° 


8° 

0°, 90°, 180°, 270° 


Table 2. FLT Reynolds number CFD solutions available for M> 1.6 


Mach Number 

Total Angle of Attack 

Aerodynamic Roll Angle 


0° 

0° (c/) a undefined for a T = 0°) 

1.6 

4° 

0°, 60°, 90°, 120°, 180°, 240°, 270°, 300° 


8° 

0° to 330° in 30° increments 


0° 

0° ((f) a undefined for a T = 0°) 

2.0 

4° 

0°, 30°, 90°, 120°, 180°, 210°, 270°, 300° 


8° 

0°, 30°, 60°, 90°, 180°, 270°, 300°, 330° 


0° 

0° (tj) a undefined for a T = 0°) 

3.0 

4° 

0°, 30°, 90°, 120°, 180°, 210°, 270°, 300° 


8° 

0° to 330° in 30° increments 


0° ((/>„ undefined for a T = 0°) 
0°, 30°, 120°, 210°, 300° 



0° 

0° ((f) a undefined for a T = 0°) 

4.5 

4° 

0°, 90°, 180°, 270° 


8° 

0°, 30°, 60°, 90°, 180°, 270°, 300°, 330° 


C. Response Surface Generation using Radial Basis Functions 

The intent of this paper is not to delve into the mathematics of radial basis function response surface methods 
(which are available elsewhere 14-16 ); instead, this paper focuses on the application of RBF algorithms to a practical 
problem. However, a short discussion of RBF methodology is presented here for clarity. In general terms, radial 
basis function networks are analogous to solving a system of equations where each input data point has an “island of 
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influence” on the global response. The relative size of each input point’s “island” determines the importance of that 
input point to the final solution, allowing the user to weight the influence of each input point. A multidimensional 
response surface modeling software package (such as the NEAR RS software from Nielson Engineering and 
Research, Inc. 16-19 used to develop the CFD-based A106 AFMA DB) constructs a global response surface across 
multiple dimensions based on a sparse set of input data. 

There are several attractive features to using RBFs to generate the response surfaces for aerodynamic databases. 
Unlike parametric regression models, there are no requirements for the response surface to show linearity or 
parametric behavior in any dimension. Thus, radial basis functions can model extremely complex behavior. 
Additionally, RBFs do not require the input data to be regularly spaced in any dimension of the domain and can be 
updated easily as new data becomes available. In the author’s experience, response surfaces constructed using RBF 
methodology are ideally suited to constructing aerodynamic databases that need to model complex behavior with 
only a relatively sparse set of input data available. 

However, the features that make RBF methodology attractive for generating aerodynamic databases also lead to 
several of the corresponding challenges in a practical sense. Constructing an RS that adequately models the 
underlying data often requires extensive user input regarding the relative size of the “island of influence” for each 
data point. In the author’s experience, this step of determining the proper “island of influence” for each input point is 
the single most important and difficult step in computing response surfaces using RBFs. Therefore, the author has 
found that this methodology is most useful in instances where the general behavior of the output response variables 
across the domain is known to some extent, so that the user can iterate the algorithm until the RS models the 
expected behavior. The CFD-based DB presented here was exactly such a circumstance, since the Ares I aero team 
had extensive knowledge of the aerodynamics of the Ares I CLV from the analyses performed during earlier design 
iterations. The A106 AFMA DB was expected to show similar general trends to the previous data. In such a 
situation, RBF methodologies are powerful tools that model the underlying data based on a reasonably sparse set of 
input data points. 


II. Development of the CFD-based Supersonic A106 AFMA DB 

The aero team developed the CFD-based Supersonic A 106 AFMA DB using the NEAR RS software package. 
The objectives of the aero team for the CFD-based DB were to match the CFD data as closely as possible, to show 
proper physical behavior (aerodynamic coefficients show periodicity in <j> a , etc), and to match the behavior expected 
based on data from previous design cycles. Discontinuities and large local slopes in the database generally present a 
challenge to the control of the vehicle, so the aero team desired the smoothest possible response surface that fulfilled 
the above goals. The flexibility of RBF response surface generation meant that the DB development was an iterative 
process, with the primary trade-off being the global smoothness of the RS versus going through the input CFD data. 
The team used the RS identification module of NEAR RS to compute a global RS across the desired domain of 
Mach number, total angle of attack (ay), and aerodynamic roll angle The global RS was then queried at a 

regular set of breakpoints for the independent variables. Finally, the team made some final minor adjustments to the 
queried RS to ensure mathematical consistency. 

For the CFD-based DB, separate response surfaces were generated for the longitudinal coefficients ( C rl , C v> and 
C,„) and the lateral-directional coefficients (Cy, C/, and C„) in the missile axis system because the lateral-directional 
coefficients show much more complex behavior than the longitudinal coefficients, especially versus aerodynamic 
roll angle. While the missile axis Cy and C„ in the AFMA DB have little impact, C/ was considered a key driver to 
the trajectory, guidance, and control simulations of the Ares I CLV. The aerodynamic C/ for the Ares I CLV was 
generated primarily by the interaction of aerodynamic loads on the protuberances, and the aero team observed that 
the rolling moment exhibited complex behavior across the domain throughout the Ares I aerodynamic analyses. 3 
The longitudinal response surface calculation was more robust than the lateral-directional RS; slight changes to the 
parameters or removal of input points did not affect the final longitudinal coefficient RS, while such changes had 
major effects on the lateral-directional RS. The sensitivity to changes in parameters made computing the lateral- 
directional RS while preserving all desired behaviors especially difficult. 

The DB team used 104 WT Re CFD solutions for the development and analysis of the CFD-based DB. The 
actual NEAR RS input file, however, contained more than 104 input data points. The DB team computed a simple 
linear model to adjust the final WT-based A106 AFMA DB to simulate FLT Re conditions based on CFD data at 
WT and FLT Re. 10 The DB Team also computed the inverse of that linear model and applied it to the A106 FLT Re 
CFD solutions to create “approximate” WT Re solutions. The general form of this correction is shown in Eq. (1). 

C* = YYlC' 

^ i,WT approx. rn ^i,FLT 
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(i) 



At any point in the domain where an actual WT Re CFD solution existed, the approximate solution from the FLT Re 
CFD was omitted. The rest of these approximate WT Re CFD solutions (33 total) were included in the NEAR RS 
input files. 

Unlike many parametric methods, the user cannot specify parameters such as derivatives of the response surface 
at the domain edges using RBFs. Instead, the DB team created “ghost” data points to ensure proper trends near the 
domain boundaries using the teams’ experience with the behavior of the AFMA DB for previous design iterations as 
a guide. The CFD data for Mach numbers less than 1.6 were used to ensure the proper behavior near Mach 1.6. 
Several ghost points were added at Mach 5.0 (and a few points at Mach 6.5 for the lateral-directional RS) to ensure 
the expected behavior near Mach 4.5. By definition, the aerodynamic coefficients must be periodic with 
aerodynamic roll angle (since = -30 degrees is the same as 330 degrees, etc.), so the team populated ghost data 
for aerodynamic roll angles from -90 to 0 degrees and 360 to 450 degrees using this periodicity. The data from the 
single CFD solution acquired at a total angle of attack of 0 degrees were extended across the entire aerodynamic roll 
angle domain at every 30 degrees because aerodynamic roll angle is undefined at zero degrees. The team also added 
interpolated data points at 0 degrees total angle of attack for the nominal DB Mach number breakpoints where no 
CFD data were available (1.8, 2.5, and 3.5). The DB team reflected the data from 0 to 4 degrees total angle of attack 
about 0 degrees to correctly model coefficient behavior near o.t= 0 degrees. For the same reasons, the team created 
ghost points at 10 degrees total angle of attack by extrapolating the available CFD data. Finally, the team added 
eight points at a r = 6 degrees for aerodynamic roll angles of 0 and 1 80 degrees approximated using a smoothing 
spline interpolation based on the A103 AFMA DB behavior at those conditions. 

The final CFD-based Supersonic A106 AFMA DB closely matched the available CFD data. Table 3 provides the 
computed coefficient of determination (R 2 ) statistics for the CFD-based DB evaluated at both the input CFD points 
and the DB assessment CFD points. The coefficient of determination is a standard statistical metric for evaluating 
goodness-of-fit and is often used for polynomial and other regression models. 25 A value close to unity represents 
very little quantitative difference between the model and the input points, while a value less than 0.5 in general 
represents a poorer fit. The results in Table 3 clearly show that the CFD-based DB closely models the CFD data. 
The R 2 values for the lateral-directional coefficients indicate a somewhat poorer fit than for the longitudinal 
coefficients, but the high R~ values still indicate good agreement between the CFD-based DB and the input CFD 
data. The very low R 2 value at the DB assessment points for C Y occurred because the predicted coefficient values at 
those conditions were small in magnitude, so the R 2 data are somewhat misleading for that case. 

Table 3. Coefficient of determination (R 2 ) statistics for the CFD-based DB evaluated at the 104 input WT 

Re CFD points and the four DB assessment points. 


Coefficient 

DB Input Points R 2 

DB Assessment Points R 2 

C A 

0.9993 

0.9990 

C y 

0.9472 

0.0126 

Cn 

0.9999 

0.9998 

Cl 

0.9987 

0.9130 

c,„ 

1.0 

0.9992 

c„ 

0.9903 

0.8323 


The input file for the longitudinal coefficient RS had 774 input points, and the lateral-directional RS input file 
had 786 input points. The RS was queried at the nominal DB breakpoints in Mach number (1.6, 1.8, 2.0, 2.5, 3.0, 
3.5, 4.0 and 4.5), total angle of attack (0 to 8 degrees in one degree increments), and aerodynamic roll angle (0 to 
360 degrees in 15 degree increments). The DB team made minor corrections to the data queried from NEAR RS to 
ensure exact data periodicity with aerodynamic roll angle and proper behavior at 0 degrees total angle of attack. 
Figure 2 shows an isometric surface plot of the CFD-based A106 AFMA DB for C/ at a total angle of attack versus 
Mach number and aerodynamic roll angle, with the corresponding original WT Re CFD data plotted as squares. It is 
clear from the figure that the CFD-based DB closely matches the available CFD data points and exhibits complex 
behavior with Mach number and aerodynamic roll angle. Figure 3 shows a similar plot for C A , and Fig. 4 gives the 
same plot for C N . 
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0.0 


360 


Figure 2. CFD-based Supersonic A106 AFMA DB and WT Re CFD data for C/ at a T = 8 degrees versus M 
and (/> a . 



360 


Figure 3. CFD-based Supersonic A106 AFMA DB and WT Re CFD data for Ca at a T = 8 degrees versus 

M and </> a . 
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360 


Figure 4. CFD-based Supersonic A106 AFMA DB and WT Re CFD data for Cn at a T = 8 degrees versus M 

and (/> a . 


III. Uncertainty Modeling for the CFD-based Supersonic A106 AFMA DB 

The Ares UQ team had not established a standard methodology for quantifying errors in CFD solutions when the 
CFD-based Supersonic A106 AFMA DB was developed; however, a reasonable estimate for the uncertainty of the 
CFD-based DB can be computed from information available at the time. The uncertainty model presented here for 
the CFD-based Supersonic A106 AFMA DB follows the general methodology used for the WT-based Supersonic 
A106 AFMA DB. ln The UQ team estimated uncertainties for significant sources of uncertainty in the database 
development process. In general, these sources can be split into three main categories: (1) input data uncertainty, (2) 
database modeling (DBM) uncertainty, and (3) database interpolation (DBI) uncertainty. For the CFD-based DB, the 
A106 CFD solutions are the input data, so the input data uncertainty is the uncertainty in the CFD solutions. The 
overall uncertainty model for the CFD-based DB is shown in Eq. (2). The UQ team assumed that each of the 
uncertainty sources was uncorrelated with the others. The CFD data uncertainty and the overall model have both a 
bias and random component. Some of the residuals included in the random uncertainty component may not actually 
be random errors, but the UQ team treated the residuals as random variations by pooling data across the domain. 


u 


i ,CFD DB fandom 

U, 


= Ju; 


2 + u 2 + rj 2 

CFD, random ~ ^ i, DBM ' u i,DBI 


i ,CFD DB Jbias 


^ i , CFD , bias 


( 2 ) 


A. A106 CFD Data Uncertainty 

Quantifying the uncertainty for a set of CFD solutions typically involves both a verification and validation 
(V&V) phase (see Ref. 20-22 for further discussion of V&V for CFD). Verification refers to the process of 
confirming that a computational model implementation accurately represents the underlying conceptual model and 
its solution, 22 which for CFD codes typically includes order of accuracy tests, grid convergence tests, and 
comparisons with exact or highly accurate solutions. Validation refers to the process of determining if a simulation 
is an accurate representation of the intended real-world application 2 " and typically involves comparing the simulated 
results to wind tunnel and flight test data and evaluating the accuracy of the simulation. The uncertainty model 
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developed by the UQ team for the input A106 CFD solutions is shown in Eq. (3) and reflects both aspects of V&V. 
The model contains a term constructed from code-to-code comparisons ( U^cic ) as well as both bias (E iC FD,bms) and 
random (Uivai, random) validation uncertainty terms from comparison of CFD data to WT data from the earlier Ares I 
DAC on the A103 OML. The kMi term in the model is a margin index added for conservatism because the CFD 
validation comparisons for the A106 OML may not be identical to the A103 comparisons. The UQ team selected a 
value of 1.15 for this term based on engineering judgment, which corresponds to increasing the estimated random 
validation uncertainty term by 15 percent. 


u 


i ,CFD , random 

u 


~ ^‘^i,C2C 


( ^ MI F j yal, random ) 


i , CFD .bias ^i, CFD, bias 


( 3 ) 


A classical solution verification analysis for a set of CFD solutions involves estimating the discretization error in 
the solutions through the rigorous use of grid refinement and metrics such as Richardson extrapolation. 21 That type 
of detailed analysis for each set of CFD solutions is often difficult to perform in a fast-paced vehicle design project 
because of schedule and resource limitations. The general quality of the Ares I CFD solutions was confirmed 
throughout the DAC process through the establishment and revision of best practices, 1 ’ 6 but the discretization errors 
for the A106 CFD solutions were not rigorously quantified. Instead, the UQ team merged code-to-code 
comparisons for the A106 OML with previously presented evaluations of the code-to-code differences in the CFD 
solutions 13 to establish a surrogate estimate of the discretization error in the CFD solutions. The detailed 
methodology for computing the code-to-code term will only be summarized here. At all conditions with CFD 
solutions available from more than one CFD code, the UQ team computed the range of all available data for each 
coefficient by finding the absolute value of the difference between the maximum and minimum values. The UQ 
team then computed a robust estimator of the standard deviation by dividing each range by a statistical bias factor 
(di) that depends only on the number of solutions used to compute each range. 2 ' After examining the normalized 
range ( R/d j) data, the UQ team determined that the code-to-code differences for M> 1.6 were primarily dependent 
on the total angle of attack and pooled the normalized ranges across all of the other variables. Additionally, the UQ 
team pooled the data from the A101, A103, and A106 OMLs and for simulations at both WT and FLT Reynolds 
numbers. Finally, the UQ team estimated the code-to-code uncertainty term in Eq. (3) by determining a reasonably 
smooth curve that bounded all of the normalized range data. Figure 5 shows the normalized ranges and the estimated 
code-to-code uncertainty. Each data point in the figure corresponds to the normalized range of data from all codes at 
a unique combination of OML and Re (i.e., “A101 WT” denotes the data from the A101 OML at WT Re). 
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Figure 5. Normalized range data for code-to-code comparisons of Ares I CFD solutions for the A101, 
A103, and A106 OMLs at WT and FLT Re atM > 1.6 and estimated code-to-code uncertainties versus a T . 

The other two terms in Eq. (3) arise from validation of the Ares I CFD data with experimental results. The Ares 
I aero team acquired the A 106 CFD solutions prior to the wind tunnel test in accordance with Ares I CFD best 
practices, 1 so the UQ team used computational and experimental data for a previous design, the A103 OML, to 
estimate the validation uncertainties. The A103 AFMA DB was based entirely on WT data for the A103 OML, with 
corrections made to remove experimental bias errors. Figure 6 shows a plot comparing the A103 CFD and AFMA 
DB data at M = 1.6 and a r = 7 degrees versus aerodynamic roll angle. In the figure, the CFD data are shown as red 
circles with the estimated code-to-code uncertainty shown as error bars, the DB data are shown as the blue solid line 
with diamond markers, and the shaded blue area bounded by the blue dashed lines represents the DB uncertainty 
intervals. 24 For each available A103 CFD solution (74 total for M> 1.6), the UQ team computed a validation 
residual by subtracting the A103 AFMA DB data from the corresponding CFD data. The UQ team divided the CFD 
validation uncertainty into both a random and a bias term because the Ares aero team observed a consistent bias 
between the USM3D CFD solutions and WT data for the longitudinal coefficients over the course of the project 
(e.g., USM3D consistently predicted CAPF to be lower than the measured wind tunnel data). The UQ team 
concluded that the validation residuals depended primarily on the total angle of attack, so the team pooled the 
residuals for all Mach numbers and aerodynamic roll angles together at each total angle of attack. 
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Figure 6. Comparison of A103 CFD data to the A103 AFMA DB at M = 1.6 and a T =l degrees versus tj> a 


The UQ team estimated the bias term, denoted E iC FD,bms in Eq. (3), by finding the median of the validation 
residuals at each total angle of attack and then making some slight adjustments using engineering judgment. The 
team assumed that the bias term was zero for the lateral-directional coefficients because the data did not suggest that 
a consistent bias existed across the entire domain. For the random component of the validation uncertainty, denoted 
Ui^ai, random in Eq. (3), the UQ team subtracted the estimated bias uncertainty term from the validation residuals, 
resulting in a set of bias-corrected residuals. The team computed normalized ranges from the bias-corrected 
residuals and set the final estimate of the random uncertainty term equal to three times the mean normalized range. 
The team again made some minor adjustments to ensure smoothness. Figure 7 shows the A 103 validation residuals 
(CFD minus WT DB data), the estimated bias uncertainty term (the solid purple line with square markers), and the 
random uncertainty term (the dotted purple bounds about the bias). The uncertainty intervals for the A103 AFMA 
DB 24 are shown as dashed red lines in the figure for reference and are symmetric about zero since a validation 
residual equal to zero indicates no difference from the DB value. 
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Figure 7. CFD-to-DB residuals for the A103 AFMA DB at M > 1.6 and estimated CFD validation bias and 
random uncertainties versus a T . 


Table 4 lists the computed estimates for the code-to-code uncertainty and the bias and random terms of the CFD 
validation uncertainty at 8 degrees total angle of attack, reported as percentages of a reference value for each 
coefficient. The reference values for each coefficient are approximately the maximum magnitude for that coefficient 
across the entire A106 CFD-based AFMA DB. The results in Table 4 clearly show that the random uncertainties for 
the lateral-directional coefficients are much larger relative to the reference values than for the longitudinal 
coefficients. This is largely due to the fact that nominal DB values of the lateral-directional coefficients are much 
smaller than nominal values of the longitudinal coefficients. For instance, the normal force coefficient magnitudes 
are typically on the order of 10 times larger than the side force coefficient magnitudes at 8 degrees total angle of 
attack. The lateral-directional coefficients for the Ares I CLV are driven by complex phenomena such as vortex 
shedding, the interaction of wakes from various protuberances on other protuberances, and the vehicle boundary 
layer behavior; so the results in Table 4 are not surprising. 


Table 4. CFD uncertainty estimates for the code-to-code, validation bias, and validation random terms 
as a percent of reference values at a T = 8 degrees. 


Coefficient 

Code-to-Code 

Validation Bias 

Validation Random 

C A 

0.96% 

-1.80% 

3.09% 

C y 

31.9% 

0% 

154% 

CV 

2.39% 

3.50% 

3.30% 

Ci 

13.3% 

0% 

41.4% 

C m 

1.82% 

2.57% 

3.52% 

C n 

28.9% 

0% 

165% 


The UQ team also used a validation metric called the average relative error metric proposed in Ref. 21 to 
quantify the accuracy of the A 103 CFD predictions relative to the corresponding A 103 WT -based DB. This global 
metric is a measure of the average difference between the CFD and WT data relative to the WT data values at each 
point. The UQ team slightly modified the metric presented in Ref. 21. Since the lateral-directional coefficients for 
the Ares I CLV cross zero throughout the aerodynamic roll angle domain at a given total angle of attack, the UQ 
team did not use the mean DB data at each condition in the denominator of the metric as formulated in Ref. 21. The 
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team instead used the maximum A 103 AFMA DB magnitude across the entire aerodynamic roll angle domain at 
each given Mach and total angle of attack. This formulation for the mean relative error metric is shown in Eq. (4), 
where E is the difference (or error) between the CFD and WT data for each solution, C lre f„ is the reference 
maximum value from the A 103 AFMA DB for that condition, N is the number of available CFD validation points, 


and 



I mean 


is the mean relative error metric. 
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1 ^ jV 
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It is useful to compute median and maximum relative error metrics in addition to the mean relative error metric. 
The median relative error metric uses the same general equation above, and the maximum relative error metric 
corresponds to the maximum value of the term inside the summation on the right-hand-side of Eq. (4) at any of the 
CFD conditions. It is necessary to omit any points where the reference value is very close to zero to avoid artificially 
inflated the results of these metrics. For this computation, the UQ team omitted the points at 0 degrees total angle of 
attack for all coefficients except Ca because the reference value magnitudes are very small for those conditions. To 
help interpret the significance of the mean relative error metric, the UQ team also normalized the WT DB 
uncertainty by the same reference data to compute a mean relative uncertainty (analogous to the mean relative 
confidence indicator in Ref. 21). Table 5 gives the mean, median, and maximum relative error metrics for the A103 
CFD as well as the mean relative uncertainty. These metrics show that the relative errors for the CFD predictions of 
the lateral-directional coefficients are much higher than for the longitudinal coefficients, but the correspondingly 
higher mean relative uncertainties indicate that the WT DB uncertainties are also larger relative to the lateral- 
directional coefficients. The large difference between the mean/median relative error and the maximum relative 
error metrics for the lateral-directional coefficients is also an indication that the CFD data may not adequately model 
certain behaviors in the WT data for those coefficients. The mean and median relative error metrics for all 
coefficients are less than the mean relative uncertainty from the A103 DB, but this is only a comparison of the mean 
values across the domain and does not indicate that the CFD-to-WT residuals are within the DB uncertainty intervals 
at any given point. 

Table 5. Validation metrics for A103 CFD solutions and the mean relative A103 AFMA DB uncertainty. 



CFD Validation Metrics 
(Relative Error in Percent) 

Relative DB 
Uncertainty 

Coefficient 

Mean 

Median 

Maximum 

Mean 

C A 

3.60% 

3.26% 

7.53% 

6.06% 

C y 

27.8% 

26.1% 

94.9% 

94.9% 

Cn 

6.26% 

6.00% 

13.2% 

11.9% 

c, 

17.0% 

14.3% 

66.6% 

38.6% 

C,„ 

4.09% 

4.17% 

8.50% 

10.4% 

c„ 

24.4% 

20.4% 

97.0% 

86.0% 


B. Database Modeling Uncertainty 

The database modeling uncertainty for the CFD-based DB accounts for errors incurred by the process of taking 
the input CFD data and constructing the final DB. The UQ team evaluated if the NEAR RS output modeled the input 
CFD data by comparing the RS computed in NEAR RS to the input CFD data points. The prediction capability of 
the response surface was evaluated by comparing the RS computed in NEAR RS to the four CFD points obtained for 
database assessment in the domain that were not used to develop the DB. In both cases, the DBM uncertainties were 
only considered significant if the residuals between the RS and the actual CFD data fell outside of the estimated 
discretization error bounds of the CFD data, which were assumed to be the estimated code-to-code uncertainties. 

The UQ team computed the range between the CFD-based DB values and the original CFD data at each of 
the 104 original WT Re CFD conditions and plotted the residuals versus total angle of attack. All of the residuals 
were within the estimated code-to-code uncertainty bounds. The same procedure was performed for the four DB 
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assessment points, and again the UQ team observed that all of the residuals were within the estimated bounds. 
Therefore, no additional uncertainty for database modeling errors was included in the uncertainty model for the 
CFD-based DB because the observed database modeling errors were within the input data uncertainty intervals. 
Therefore, the UQ team set the U i D / iM term in Eq. (2) equal to zero. 

C. Database Interpolation Uncertainty 

The database interpolation uncertainty accounts for any errors that occur as a result of a database user querying 
the discrete database at any given condition within the domain. The implementation instructions for the CFD-based 
DB directed the user to linearly interpolate from the discrete database conditions to any condition in the domain, so 
the DBI uncertainty is directly related to the non-linearity of the data between the reported discrete conditions. To 
evaluate this uncertainty, the UQ team created a finely spaced set of query points across the domain and queried the 
final CFD-based DB at those points using both a linear interpolation and a higher-order cubic spline interpolation. 
The team evaluated the residuals between the two interpolation methods using the same methodology used for the 
DBM uncertainty. As with the DBM uncertainty, the UQ team observed that all of the residuals were within the 
estimated bounds and therefore set the Uiobi term in Eq. (2) equal to zero. 

IV. WT-based Supersonic A106 AFMA DB and Uncertainty 

The final WT-based Supersonic A106 AFMA DB developed from the UPWT Test 1972/1843 data provided the 
aerodynamic coefficients and uncertainties at the flight Reynolds number, using a wind-tunnel-to-flight Reynolds 
number correction derived from CFD. The full description of the development of the final A106 AFMA DB is 
provided in Ref. 26, while the details of the uncertainty analysis are presented in Ref. 10. This paper will briefly 
summarize the general procedures and highlight some modifications to the data made for the current comparisons, 
but the reader is directed to the reference documents for additional details. To ensure a valid comparison, this paper 
does not use the final WT-based Supersonic A106 AFMA DB as released, but instead uses the DB for the baseline 
A106 OML at WT Re developed directly from the UPWT Test 1972/1843 data. 

A. Database Development Summary 

The DB team constructed the WT-based DB for the baseline A106 OML using a procedure developed during the 
previous DACs (DAC-2A and DAC-2B) to remove systematic errors from the wind tunnel data. 26 An axisymmetric 
model (designated the C4 configuration) of the A 106 OML with all protuberances removed was tested in UPWT 
Test 1972/1843, and the DB team used this data to remove systematic errors associated with the wind tunnel from 
the baseline A106 configuration (Cl) data. Both the axisymmetric configuration and the baseline configuration 
were tested using pitch sweeps at fixed aerodynamic roll angles and roll sweeps at fixed total angles of attack. The 
overall construction of the WT-based DB for the baseline A106 OML is defined in Eq. (5). 

Ci,DB = {^i, Cl. pitch — Cj'CA, pitch) + Q,C4 adjusted .pitch + ^i,C\,roll ^ 

The DB team first constructed “corrected” C4 pitch sweep data (denoted C u c 4 adjusted, pitch) at 0 degrees aerodynamic 
roll angle by smoothing the C4 WT pitch sweep data and invoking symmetry or anti-symmetry about 0 degrees 
angle of attack for the longitudinal coefficients (Cj, C-n and C,„), since an axisymmetric model would be expected to 
show symmetry (for C,) or anti-symmetry (for C v and C m ) about 0 degrees. The lateral-directional coefficients (C>, 
Ci and C„) for the “corrected” C4 data were set to zero. Next, the DB team computed protuberance increments at 0 
degrees aerodynamic roll angle by subtracting the measured C4 pitch sweep data (denoted C i:C 4 , pud,) from the Cl 
pitch sweep data (denoted C^ci, pitch)- Finally, the team calculated incremental coefficients due to aerodynamic roll 
angle dependence (denoted AC, c;, TO «) using the Cl WT roll sweeps (available at total angles of attack of 0, 2, 4, 6, 
and 8 degrees) by subtracting the 0 degree values of the coefficients from the rest of the roll sweep. This process 
resulted in a database with independent variables of Mach number, total angle of attack, and aerodynamic roll angle. 

B. Uncertainty Modeling Summary 

The details of the uncertainty modeling for the baseline A 106 AFMA DB are described in Ref. 10 and will not 
be repeated here. The overall uncertainty buildup for the DB at WT Re contains three terms: 1) the experimental 
uncertainty based on the wind tunnel repeatability, 2) the database modeling uncertainty for creating the baseline 
DB, and 3) the database interpolation uncertainty. The experimental uncertainty for the WT-based DB presented in 
Ref. 10 depends only on the estimated data repeatability, which the UQ team computed using data from both the 
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A106 test in the UPWT as well as data from previous Ares I WT tests in UPWT. Replicate runs were performed 
throughout each WT test, often with several days or significant configuration changes occurring between replicates, 
so the repeatability estimated by the UQ team could also be referred to as reproducibility based on the nomenclature 
of the US Guide to the Expression of Uncertainty in Measurement f 1 The UQ team assumed that the DB 
development process corrected for any bias errors present in the WT data, and the estimated repeatability for the 
WT-based DB characterized all of the remaining variability in the wind tunnel environment. The final computed 
uncertainties for the A106 AFMA DB are much larger relative to the nominal data for the lateral-directional 
coefficients than for the longitudinal coefficients. However, the actual magnitude of the CV and C v uncertainties are 
identical, and the same is true for C m and C„. Throughout the Ares DAC process the UQ team generally found that 
the Ci uncertainties were quite large relative to the coefficient values; however, that observation was not true for the 
WT-based Supersonic A106 AFMA DB uncertainty. 

V. Assessment of CFD-based Supersonic A106 AFMA DB 

The primary goal of this paper is to assess the quality of the CFD-based Supersonic A 106 AFMA DB with 
respect to the final WT-based DB that was released. This assessment includes both a qualitative comparison of the 
general trends of the data with respect to the independent variables and a quantitative assessment of the values of the 
aerodynamic coefficients in both DBs. Since there are no formally defined requirements on the necessary accuracy 
of aerodynamic data for the Ares project, there is no way to definitively conclude whether the CFD-based DB is 
adequate relative to the WT-based DB. A general assessment will be made, however, by examining the qualitative 
comparisons of the data and the DB-to-DB residuals with respect to the computed uncertainty bounds. 

There are striking qualitative similarities between the WT- and CFD-based DBs. Figure 8(a) shows an isometric 
surface plot of the WT-based DB for C/ at 8 degrees total angle of attack versus Mach number and aerodynamic roll 
angle, corresponding to the same plot as Fig. 2 for the CFD-based DB. The CFD-based DB plot from Fig. 2 is 
repeated in Fig. 8(b) to aid in visually comparing the two DBs. Clearly, the general trends and behaviors of the DBs 
are very similar across the domain shown. This general qualitative similarity is true for all other aerodynamic 
coefficients in both DBs. 
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Figure 8(a). WT-based A106 AFMA DB for C/ at a T = 8 degrees versus M and <f> a . 
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Figure 8(b). CFD-based Supersonic A106 AFMA DB and WT Re CFD data for Q at a T = 8 degrees versus 
M and (f> a (repeated from Fig. 2 for comparison). 
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Figure 9 shows line plots of the CFD-based and WT -based DBs versus total angle of attack at Mach 3.0 and an 
aerodynamic roll angle of 90 degrees, while Fig. 10 shows the same comparison at Mach 2.0 and 285 degrees 
aerodynamic roll. These two conditions were indicative of the best and worst qualitative comparisons, respectively, 
of the DB trends with respect to total angle of attack. For all coefficients except Cy and C„, it is clear that both DBs 
exhibit very similar behavior with respect to total angle of attack in both cases. However, for the side force and 
yawing moment coefficients, Fig. 10 shows that at certain conditions the trends diverge between the databases, but 
generally the agreement is reasonable. Figure 11 compares the DBs at 8 degrees total angle of attack at Mach 1.6 
versus aerodynamic roll angle and clearly shows that the WT-based DB Cy and C„ data exhibit more oscillatory 
behavior versus aerodynamic roll angle than the CFD-based DB data. This is generally true throughout the DB. 
Additionally, Fig. 1 1 shows that while the C/ trends in the two DBs are similar, there are noticeable differences 
between the DBs, especially for aerodynamic roll angles above about 240 degrees. In all of the qualitative 
comparison plots, the black dotted lines with circle markers denote the CFD-based DB, the shaded grey regions 
bounded by dashed black lines show the CFD-based DB uncertainties, the solid green lines with square symbols 
depict the WT-based DB, and the green shaded areas bounded by green dash-dot lines are the WT-based DB 
uncertainties. Each grid section in the figures is the same size for Cy and Cy and C m and C„. 


* CFD-based DBfJCFD DB Unc. —WT-based DBMWT-based DB Unc. 


LL 

CL 




01 23456789 10 

ALPHAT 



01 23456789 10 

ALPHAT 


Figure 9. Comparison of CFD-based and WT-based A106 AFMA DBs and uncertainty bounds at M = 3.0 
and <j>„ = 90 degrees versus a T - 
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Comparison of CFD-based and WT-based A106 AFMA DBs and uncertainty bounds at M 
285 degrees versus a T . 
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Comparison of CFD-based and WT-based A106 AFMA DBs and uncertainty bounds at M 
= 8 degrees versus (j> a . 
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The comparison plots in Fig. 9-11 also illustrate the differences in the uncertainties for the two DBs. The 
uncertainties for the WT -based DB depend only on Mach number and change little throughout the entire domain, 
while Fig. 9 and 10 clearly show that the CFD-based DB uncertainties vary with total angle of attack, especially for 
Cy and C n . This is due mostly to the differences in the input data uncertainties; the experimental uncertainty term in 
the WT-based DB uncertainty is based solely on the experimental repeatability, which was estimated as a single 
value for each Mach number. The CFD-based DB input data uncertainty is based on the code-to-code comparisons 
and validation comparisons, both of which are much smaller near zero degrees than for higher values of total angle 
of attack. The estimated CFD-based DB uncertainties are larger than the WT-based DB uncertainties at the higher 
total angles of attack for all coefficients. However, at zero degrees total angle of attack, the CFD-based DB 
uncertainties are actually smaller than the WT-based DB estimates for all coefficients except Ca and C/; for those 
two coefficients, the CFD-based DB uncertainties are larger throughout the domain. Although it is not clearly 
visible in Fig. 9-11, the CFD-based DB uncertainties are offset from the nominal DB values for the longitudinal 
coefficients due to the bias term from the CFD validation uncertainty, while the WT-based DB uncertainties are 
symmetric about the nominal DB. An alternative to presenting this offset uncertainty would be to add the bias term 
from the CFD validation comparison to the nominal CFD-based DB itself, which would result in symmetric bounds. 

Perhaps the most important metric for assessing the adequacy of the CFD-based DB is to quantify whether the 
true values of the aerodynamic data at WT Reynolds number are contained by the estimated uncertainty intervals. 
However, since the true values cannot be known, the UQ team instead evaluated whether the values predicted by the 
WT-based DB are contained by the uncertainty intervals estimated for the CFD-based DB. The UQ team plotted the 
DB-to-DB residuals (CFD-based DB minus WT-based DB) and then compared the residuals to the estimated 
uncertainty bounds. Figure 12 shows plots of the DB-to-DB residuals against total angle of attack along with the 
CFD-based DB uncertainty intervals shown as red dashed lines. It is clear from the figure that a large majority of 
the residuals are contained by the estimated uncertainties. Table 6 reports the percentage of the DB points that fall 
within the estimated CFD-based DB uncertainty intervals for each coefficient. For all coefficients except Cy, at least 
98.4 percent of the DB-to-DB residuals fall within the uncertainty intervals, but 92.3 percent of the C Y residuals still 
fell within the estimated CFD-based uncertainty intervals. Furthermore, Fig. 12 shows that the residuals that do fall 
outside of the CFD-based DB uncertainty intervals are generally quite close to the bounds of the uncertainty. Since 
the WT-based DB values are also uncertain, it is useful to also compare the DB-to-DB residuals to a combined 
uncertainty. Table 6 shows that comparing the residuals to a combined uncertainty computed by root-sum-squaring 
the random component of the CFD-based DB uncertainty and the WT-based DB uncertainty increases the 
percentage of points within the bounds to 100 percent for all coefficients except rolling moment, where the percent 
within the bounds is 99.6 percent. Both of these metrics indicate that the CFD-based DB and estimated uncertainties 
do an excellent job of predicting the actual aerodynamic coefficients of the WT-based DB, which was the original 
purpose of the CFD-based DB. 

Table 6. Percentage of DB-to-DB residuals for the CFD-based and WT-based DBs within the uncertainty 

intervals estimated for the CFD-based DB and for a combined DB uncertainty. 



Percentage of DB-to-DB Residuals Within Interval 

Coefficient 

CFD-based DB Uncertainty 

Combined Uncertainty 

C A 

100% 

100% 

Cy 

92.3% 

100% 

Cn 

99.9% 

100% 

c, 

99.4% 

99.6% 

c,„ 

98.4% 

100% 

c„ 

99.4% 

100% 
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Figure 12. DB-to-DB residuals for the CFD-based and WT-based Supersonic A106 AFMA DBs and the 
estimated CFD-based uncertainty intervals versus a T . 


Another way to quantify the differences between the two databases is to apply the validation metrics discussed 
earlier, such as the mean relative error metric, to the DB-to-DB residuals. Table 7 shows the mean, median, and 
maximum relative error metrics for the DB comparison as well as the mean relative uncertainty computed using the 
combined uncertainty discussed above. As for the CFD validation computations, the reference values for each 
coefficient are the maximum magnitude of the CFD-based DB at the given Mach number and total angle of attack 
and conditions, and the points at 0 degrees total angle of attack were omitted for all coefficients except Ca . The 
relative errors for the lateral-directional coefficients are quite large, especially the maximum relative error. As noted 
for the CFD validation comparison, the large values of the relative error and mean relative uncertainty for the 
lateral-directional coefficients is primarily because the reference values for those coefficients are quite small 
throughout the database. The mean magnitudes of the DB-to-DB residuals themselves are actually smaller for C Y 
and C„ than for C N and C m , respectively. 


Table 7. Relative DB-to-DB error metrics for the CFD-based and WT-based A106 AFMA DBs and the 
mean relative combined uncertainty. 



DB-to-DB Relative Error Metrics 
(Relative Error in Percent) 

Relative 

Uncertainty 

Coefficient 

Mean 

Median 

Maximum 

Mean 

C A 

4.50% 

4.44% 

10.3% 

6.65% 

C y 

51.6% 

40.1% 

326% 

238% 

Cjv 

5.75% 

4.92% 

30.2% 

16.7% 

Ci 

33.2% 

24.1% 

187% 

158% 

C m 

4.16% 

3.02% 

28.2% 

16.4% 

c„ 

50.0% 

37.0% 

442% 

389% 


It is also important to evaluate whether the CFD validation uncertainty estimates developed by the UQ team from 
the A103 CFD data do an adequate job of modeling the differences between the A106 CFD and WT data. This post- 
facto comparison was not possible at the time the CFD-based DB was released because the WT data for A106 were 
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not available at that time. However, this evaluation is an important step in confirming the adequacy of the 
uncertainty model for the CFD-based DB presented in this paper. Figure 13 shows the A106 CFD data (as red 
circles with error bars from the code-to-code uncertainty term) and the A106 WT -based DB (green lines with square 
markers) and uncertainty intervals (shaded green regions bounded by dash-dot lines) versus aerodynamic roll angle 
at the same conditions shown for the A103 CFD in Fig. 6 (Mach 1.6 at a total angle of attack of 7 degrees). The 
qualitative differences between the CFD and WT DB for A103 and A106 appear to be very similar when comparing 
Fig. 13 to Fig. 6. Figure 14 shows the A106 CFD validation residuals (CFD data minus WT -based DB values) in 
comparison to the estimated A 103 validation uncertainties. The estimated A 103 validation bias term is shown as a 
dashed purple line and the estimated A 103 validation random term bounds are shown as purple dotted lines that are 
symmetric about the bias. The estimated A103 uncertainty bounds contain most of the A106 residuals, and the 
estimated A 103 validation biases appear reasonable for the A 106 data as well. There are a few A 106 residuals 
outside of the A103 uncertainty estimates, especially for C Y and C„ near 0 degrees and C N and C m above 4 degrees 
total angle of attack. The good general agreement between the A 103 estimates and the A 106 data confirms that the 
relationship between the CFD and WT data did not change significantly between design cycles for this data set, and 
this analysis provides confidence in the general methodology of using validation comparisons from previous design 
cycles to estimate validation uncertainty for the Ares I Supersonic AFMA DB. The excellent agreement between 
the validation biases for A103 and A106 suggests that instead of including the bias term in the uncertainty, the DB 
team could have applied the estimated biases as a correction to the CFD-based DB itself. 
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Figure 13. Comparison of A106 CFD data to UPWT data and the WT-based DB uncertainty bounds at 

M = 1.6 and a T =l degrees versus (/>„. 
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Figure 14. A106 CFD-to-WT residuals for the A106 CFD data at M > 1.6 and estimated A103 CFD 

validation bias and random uncertainties versus a T . 


VI. Conclusions 

Although the CFD-based and WT-based Supersonic A106 AFMA DBs were developed independently and the 
input CFD and UPWT data were not adjusted in any way, the final resulting DBs exhibit qualitatively similar 
behavior throughout most of the domain, especially for the longitudinal aerodynamic coefficients. The two DBs do 
show diverging results for C Y and C„ at some conditions, but the majority of the differences are within the 
uncertainty intervals for the CFD-based DB. The intended purpose of the CFD-based DB was to predict the 
aerodynamic coefficients in the WT-based DB before the WT data was available, and the quantitative comparison 
between the DBs suggests that the CFD-based DB did an admirable job at this purpose. Although the uncertainty 
intervals for the CFD-based DB presented here were not provided at the time of release because of schedule 
constraints and the methodology’s immaturity at the time, all of the data needed to estimate the CFD-based DB 
uncertainties were available at the time of release. The estimated uncertainties for the two DBs are quite different in 
character, and the CFD-based DB uncertainties are generally larger than the WT-based DB except at low total 
angles of attack. 

The assessment of the CFD-based DB presented in this paper suggests that the methods used by the Ares aero 
team both in acquiring the A106 CFD data and developing the CFD-based DB produced reliable and reasonably 
accurate results given the estimated uncertainty intervals. This is significant because at the time of the release of the 
CFD-based DB, there was considerable concern as to whether CFD could be used without the use of any WT data to 
construct such a DB for the Ares I CLV. The current analysis shows that the methodology used to develop the 
CFD-based DB could be used in future applications, provided that certain information about the CFD analyses and 
general behavior of the data are available and the CFD test matrix and DB development strategy are designed with 
response surface generation in mind. The DB team found that knowledge of the behavior of the earlier Supersonic 
A 103 AFMA DB was critical to the development of the CFD-based DB for A 106 because the team needed that 
knowledge in order to properly fill gaps in the CFD test matrix and ensure reasonable behavior of the aerodynamic 
coefficients. Additionally, it was essential that the Ares aero team conducted the A 106 CFD simulations by 
following an established set of best practices that were also used for the earlier A103 CFD simulations. This 
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continuity in the analysis methodology allowed the UQ team to assume that the A 103 validation estimates could be 
used for the A106 OML. The present analysis confirmed that this assumption was reasonable because the validation 
residuals for both the A103 and A106 OMLs were similar. 
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